The case for reconfigurable I/O channels

Datacenter environments are increasingly layered, with multicore parallelism, OS virtualisation and NUMA memory all introducing variable latency and throughput for data transmission. For a programmer deploying applications in such a shifting environment, it is unclear how best to use venerable interfaces such as the sockets layer. Kernel hackers realise there is some performance hit to all the software layering, but quantitative figures are hard to find. This is a position paper of two rather different halves. We first seek to understand just how big the impact of NUMA layouts and OS virtualisation have been on I/O performance. To do this, we implemented a variety of IPC mechanisms (from TCP sockets to shared memory) and benchmarked them under modern multicore hardware and Xen. We discovered a large variance in throughput and latency under different scheduling conditions (over an order of magnitude in some cases), and also some rather inexplicable results which point to the extreme difficulty of predicting crosslayer performance. In the second half, we describe the early design of a system which aims to overcome these multiplexed I/O scheduling issues. It provides an efficient, zero-copy data transmission interface that automates the selection of the underlying transport, and the facility to dynamically reconfigure transports as system conditions change. Finally, we discuss the implications of extending the OS with explicit I/O flow tracking—eliminating contention, transparent transport-level security and an upgrade path to multi-path TCP.